Lenient or Strict Application of IRT with an Eye on Practical Consequences

Item Response Theory (IRT) is a class of measurement models that offers a variety of methods, not only to measure latent properties, but also to assess and improve the quality of such measurement. The attractive properties of IRT models, however, can only be derived when the rather restrictive assumptions of the model hold for the data being considered. Lenient researchers will argue that the main conclusions are fairly robust against minor or moderate violations of the assumptions. Strict researchers, on the other hand, may wish to modify either the set of items or the model until there is good agreement between data and model. This paper will argue that several strategies with a varying degree of leniency will usually lead to the same major substantive conclusions, although the results may differ in some minor aspects. This will lead to some recommendations based on the pros and cons of being lenient and being strict.